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ABSTRACT 


The  Combined  Arms  Analysis  Directorate  of  the  Training  and  Doctrine  Analysis  Command 
(TRAC)  uses  the  Analytic  Hierarchy  Process  (AHP)  to  evaluate  the  contribution  of  modernization 
initiatives  to  U.S.  Army  capabilities.  This  thesis  identifies  several  problems  with  using  AHP. 
Most  significantly,  AHP  can  cause  rank  reversal  of  alternatives  if  a  new  alternative  is  considered, 
even  if  the  new  alternative  has  the  same  attribute  levels  as  one  of  the  previous  alternatives.  This 
thesis  proposes  several  modifications  that  would  improve  results  when  AHP  is  used.  It  contains 
a  different  method  of  weight  fitting  that  appears  to  provide  alternative  weights  that  are  more 
accurate  than  the  traditional  AHP  eigenvalue  method.  This  thesis  has  two  proposals  for 
improving  the  nine  point  integer  scale  by  which  pairwise  comparisons  are  made.  Most 
significantly,  this  thesis  proposes  a  modification  to  AHP  that  will  maintain  a  ratio  scale  and  avoid 
rank  reversals.  This  last  improvement  requires  the  decision  maker  to  establish  and  maintain  units 
of  measurement.  Additionally,  the  decision  maker  must  make  comparisons  of  attributes  to 
establish  a  meaningful  scale  not  sensitive  to  the  abundance  or  lack  of  alternatives  considered. 
If  units  are  maintained  and  the  decision  maker  is  consistent  in  the  pairwise  comparisons,  there 
will  be  no  rank  reversals. 
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EXECUTIVE  SUMMARY 


The  Combined  Anns  Analysis  Directorate  of  the  Training  and  Doctrine  Analysis  Command 
(TRAC)  uses  the  Analytic  Hierarchy  Process  (AHP)  to  evaluate  the  level  of  future  U.S.  Army 
modernization  that  is  achieved  by  management  decision  packages. 

A  study  of  AHP  shows  it  contains  several  flaws.  First,  and  of  greatest  concern,  AHP  can 
allow  rank  reversals  among  alternatives  if  another  alternative  is  considered,  or  if  one  is  removed. 
This  can  occur  even  if  the  additional  alternative  has  the  same  attribute  levels  a.  one  of  the  original 
alternatives. 

We  use  a  decision  problem  which  evaluates  tanks  to  illustrate  AHP's  greatest  flaw.  Assume 
that  a  tank  is  measured  by  two  attributes,  survivability  and  firepower,  and  that  the  decision  maker 
determined  firepower  is  1 .25  times  as  important  as  survivability.  To  maintain  that  preference  and 

normalize  weights  to  sum  to  one,  weights  must  be  C^>|)  for  survivability  and  firepower, 

respectively. 

Tank  characteristics: 

(1)  Survivability.  On  comparing  three  types  of  armor,  reactive  armor  is  preferred  1 .5 
over  applique,  and  applique  is  preferred  2  over  rolled  homogeneous  armor. 

(2)  Firepower.  A  tank  with  a  130mm  main  gun  is  preferred  1.4  over  a  tank  with  a 
120mm  main  gun.  A  120mm  main  gun  is  preferred  1.5  over  a  105mm  main  gun.  A  105mm  main 
gun  is  preferred  2.0  over  a  90mm  main  gun. 
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Initially,  we  compare  two  tanks  using  AHP.  Tank  1  has  applique  armor  and  a  120mm  main 


gun.  Tank  2  has  reactive  armor  and  a  lOSmm  main  gun. 


ATTRIBUTES 

RELATIVE 

TANK 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

VALUE 

TANK  1 

0.4 

0.6 

0.511 

TANK  2 

0.6 

0.4 

0.489 

figur^^TompansoiMJnwcrtanE^IsIn^AJIF?™1™^^^™ 

The  relative  value  says  one  would  prefer  Tank  1  to  Tank  2.  That  is 


XafiLJ-iLXailLJ  • 

In  addition  to  the  above  two  tanks,  we  now  consider  a  third  tank.  Tank  3  has  rolled 
homogeneous  armor  and  a  130mm  main  gun.  The  result  of  this  comparison  is  disturbing. 


TANK 

ATTRIBUTES 

RELATIVE  1 

VALUE  8 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

0.333 

0.326 

0.329 

TANK  2 

0.500 

0.217 

0.343 

TANK  3 

0.167 

0.457 

0.328 

Figure  2.  Comparison  of  three  tanks  using  AHP. 

Even  though  the  decision  maker  did  not  change  relative  weights  of  firepower  or  survivability,  or  the 
characteristics  of  the  first  two  tanks,  AHP  tells  the  decision  maker  his  preferences  have  changed. 
That  is 

Our  modification,  illustrated  using  the  same  tank  problem  described  above,  solves  the  problem 
of  rank  reversal.  In  our  modification,  the  survivability  attribute  of  Tank  1  is  selected  as  the  least 
preferred  alternative  in  survivability.  The  firepower  attribute  of  Tank  1  must  remain  1.3  times  the 
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magnitude  of  the  suvivability  attribute.  The  firepower  attribute  of  Tank  2  (105mm  main  gun)  is 
determined  to  be  equivalent  to  the  survivability  attribute  of  Tank  1  (applique  armor).  These 
restrictions  are  essential  to  maintaining  a  ratio  scale. 


TANK 

ATTRIBUTES 

RELATIVE 

VALUE 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

1.0 

1.5 

1.2778 

TANK  2 

1.5 

1.0 

1.2222 

F^ul^ 

The  result  is  consistent  with  the  result  from  using  AHP  in  the  original  problem.  That  is 


Tank  1  >  Tank  2. 

Now,  we  include  the  third  tank,  Tank  3. 


TANK 

ATTRIBUTES 

RELATIVE 

VALUE 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

1.0 

1.5 

1.2778 

1  TANK  2 

1.5 

1.0 

1.2222 

|  TANK  3 

0.5 

2.1 

1.3889 

with  additional  alternative  included,  Tank  3. 

We  maintain  the  notional  "units"  from  the  two  tank  problem.  That  is.  Tank  1  survivability  is  the 
"unit”  for  survivability  and  the  equivalency  of  Tank  1  ’s  survivability  is  maintained  with  the  value  of 
the  firepower  of  Tank  2.  The  result  of  this  consistency  in  "units"  is  the  preference 

Tank  3  >  Tank  1  >  Tank  2. 

There  is  no  rank  reversal. 

Also,  AHP  uses  an  integer  nine  point  scale,  with  reciprocals  possible,  that  severely  limits  the 
decision  maker’s  ability  to  evaluate  possible  alternatives  and  attributes  in  a  decision  problem. 
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Finally,  AHP  uses  the  eigenvalue  method  of  weight  fitting  with  little  justification.  The  weight 
fitting  by  the  eigenvalue  method  was  consistently  (nine  cases)  less  accurate  than  a  simple  least  squares 
error  (LSE)  evaluation  method. 

Despite  the  claim  of  the  developer  of  AHP,  Thomas  L.  Saaty,  rank  reversals  are  possible  even 
if  the  new  alternative  is  not  within  10%  of  original  alternatives  in  attributes.  Since  rank  reversal  can 
occur,  AHP  cannot  maintain  a  ratio  scale,  a  claim  that  it  makes  and  that  is  desirable  when  output  of 
AHP  is  used  as  input  to  optimization  models. 

This  thesis  proposes  modifications  to  AHP  that  address  the  above  problems.  First,  it  is  clear 
that  the  restriction  on  scaling  alternatives  to  integers  with  a  maximum  value  of  nine  exacerbates  the 
problems  of  inconsistency  by  a  decision  maker.  There  are  two  proposals  in  this  area.  First,  the 
decision  maker  could  identity  the  extreme  attributes  and  extreme  alternatives  by  attribute.  Assign  the 
lowest  or  least  preferred  a  value  of  one,  and  the  highest  or  most  preferred  a  value  of  nine.  In  this 
way,  all  attributes  and  alternatives  will  be  within  the  nine  point  scale.  Alternatively,  the  decision 
maker  could  use  an  open  ended  scale. 

The  second  proposal  for  modification  of  AHP  suggests  an  alternative  weight  fitting  method. 
The  method  of  least  square  errors  (LSE)  for  weight  fitting  produces  an  exact  solution  to  alternative 
weights  if  the  decision  maker  is  perfectly  consistent,  just  as  AHP’s  eigenvalue  method  will.  The  LSE 
method  appears  to  have  an  advantage  in  the  likely  case  of  decision  maker  inconsistency. 

The  third  modification  proposal  involves  solving  the  problem  of  rank  reversals  while 
maintaining  a  ratio  scale.  The  decision  maker  can  assign  the  most  preferred  alternative  in  one 
attribute  a  value  of  one,  and  assign  all  other  alternatives  a  value  of  less  than  one.  The  second  option 
is  to  assign  the  least  preferred  alternative  in  one  attribute  a  value  of  one.  If  this  standard  is 
maintained  and  the  decision  maker  is  consistent  in  all  pairwise  comparisons,  there  can  be  no  rank 
reversal. 
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I .  BACKGROUND 


A.  PROBLEM 

The  Combined  Arms  Analysis  Directorate  of  the  Training  and 
Doctrine  Analysis  Command  (TRAC)  at  Fort  Leavenworth,  Kansas 
requested  assistance  in  determining  values  for  modernization 
initiatives.  There  is  concern  that  the  Analytic  Hierarchy 
Process  (AHP) ,  the  system  currently  used,  might  not  be  the 
best  method  to  generate  relative  values  of  modernization 
initiatives . 

B.  SITUATION 

The  United  States  Army  Training  and  Doctrine  Command 
(TRAC)  at  Fort  Leavenworth,  Kansas  developed  a  new  analytical 
decision  support  system  to  help  the  Army  develop  a 
modernization  investment  program  that  reflects  the  values  of 
its  senior  leadership.  TRAC  developed  the  Research 
Development  and  Acquisition  Alternatives  Analyzer  (RDA3)  to 
determine  values  for  candidate  modernization  initiatives  by 
mathematically  optimizing  allocation  of  resources. 

C.  CURRENT  SYSTEM 

1.  Description  of  RDA3 

The  RDA3  decision  support  system  involves  two  distinct 

phases . 
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a.  Hierarchal  As a ornament 

In  the  first  phase,  modernization  initiatives 
(MDEPs)  are  determined  through  a  hierarchal  assessment.  The 
overall  goal  (Future  Army  Modernization)  is  at  the  top  of  this 
hierarchy,  and  modernization  initiatives  can  be  several  layers 
down  this  hierarchy  through  layers  of  supporting  goals  and 
objectives,  sub-goals  and  sub-objectives.  The  goal  is  the  top 
level  of  the  hierarchy  (Future  Army  Modernization)  .  The  first 
level  of  the  hierarchy  consists  of  14  Army  Mission  Areas. 
Through  a  series  of  comparative  judgement  questions,  the 
respondent  or  decision  maker  assesses  the  importance  of  each 
Mission  Area  to  Army  modernization.  Similarly,  sub-areas  are 
assessed  for  their  contribution  to  Army  Mission  Areas.  This 
process  continues  until  the  bottom  level  of  the  hierarchy  is 
reached.  The  bottom  layer  consists  of  Modernization 
Development  Programs  (MDEPs) ,  or  in  some  cases,  increments  of 
MDEPs.  In  this  way,  the  decision  maker  will  assess  the 
contribution  of  each  MDEP  to  the  goal  of  Future  Army 
Modernization . 

b.  Mathematical  Optimization 

In  the  second  phase  of  the  RDA3  decision  support 
system,  the  values  of  the  modernization  initiatives  (MDEPs) , 
constrained  by  the  annual  funding  proposal,  are  mathematically 
optimized.  The  mathematical  optimization  incorporates  several 
decision  maker  constraints  and  goals  such  as  annual  budgetary 
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limitations,  target  mission  funding  levels,  and  funding  of 
Congressionally  or  DO D  mandated  programs.  The  mathematical 
optimization  currently  in  use  is  a  new,  enhanced  math 
programming  algorithm  developed  by  Captain  Scott  Donahue,  USA, 
in  his  thesis,  while  a  student  at  the  Naval  Postgraduate 
School  under  the  direction  of  Professor  Rosenthal. 

2 .  Advantages  of  RDA3 

The  decision  support  system,  RDA3 ,  possesses  many 
characteristics  which  contribute  to  its  value  as  a  decision 
support  system.  Specifically,  RDA3 

e  is  a  comprehensive  decision  support  system  that  considers 
every  MDEP  a  potential  modernization  candidate. 

e  can  be  executed  quickly,  with  an  estimated  time  of  just 
two  to  three  hours  for  analysts  to  examine  the  outputs 
received  from  on<"  iteration  and  formulate  revised  inputs 
that  reflect  new  options  to  be  explored. 

•  would  remain  applicable  if  a  new  math  programming 
optimization  technique  was  adopted,  or  if  a  new  method  of 
determining  the  hierarchal  structure  was  implemented. 

•  is  versatile  in  that  it  will  provide  optimal  solutions  to 
budget  decisions  whether  increasing  expenditures  or 
reducing  spending  authorization. 

•  is  highly  transportable  since  it  is  built  primarily  on  the 
GAMS  linear  programming  language. 

•  is  used  to  derive  relative  values  or  priorities. 

•  is  controlled  by  HQDA  which  provides  projected  research, 
development  and  acquisition  (RDA)  constraints. 


The  many  advantages  of  RDA3  make  the  system  of  great  value  to 
the  Army  leadership.  The  system  also  has  the  advantage  of 
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flexibility  in  the  area  in  which  this  thesis  recommends 
change:  the  hierarchal  structure. 


3.  Modernisation  Initiatives 

Factors  that  influence  the  overall  desirability  of 
modernization  initiatives: 

•  modernization  plans, 

•  field  experience, 

•  attitudes  in  the  Office  of  the  Secretary  of  Defense, 

•  cost, 

•  Congressional  attitudes, 

•  warfighting  contribution, 

•  history, 

•  business  sense, 

•  personnel  considerations. 

These  factors,  however,  are  not  necessarily  an  exhaustive 
list,  and  there  is  nothing  to  prevent  the  decision  maker  from 
reducing  the  influence  of  the  above  factors,  or  even 
eliminating  one  or  more  factors  from  consideration  entirely. 

Given  the  above  factors,  TRAC  decided  the  best 
quantitative  scale  and  measurement  process  that  captures  the 
essence  of  these  influences  is  the  Analytic  Hierarchy  Process 
(AHP) . 
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II.  THK  ANALYTIC  HIERARCHY  PROCSSS 


A.  SOURCE 

The  background  information  in  this  chapter  on  the  Analytic 
Hierarchy  Process  (AHP)  is  mainly  drawn  form  the  creator  of 
AHP,  Dr.  Thomas  L.  Saaty.  Specif ically,  the  article  that 
provides  a  description  of  how  to  use  AHP,  and  an  example  of 
using  AHP  to  buy  a  house  are  from  "How  to  make  a  decision: 
The  Analytic  Hierarchy  Process"  [Ref.  1]. 

B.  HOW  TO  STRUCTURE  A  DECISION  PROBLEM 

Arrange  the  factors  that  are  important  for  the  decision  in 
a  hierarchic  structure  descending  from  an  overall  goal  to 
criteria,  subcriteria  and  alternatives.  Include  enough  detail 
to: 

•  thoroughly  represent  the  problem, 

•  consider  environment  of  problem, 

•  identify  issues  or  attributes, 

•  identify  participants. 

The  arrangement  of  a  hierarchy  serves  two  purposes:  It 
gives  an  overall  view  of  the  relationship  and  helps  the 
decision  maker  determine  if  the  issues  in  each  level  are  in 
the  same  category  so  he  can  make  accurate  comparisons.  Verbal 
judgements  are  numerically  given  values  of  (1,3, 5, 7, 9)  ranging 
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from  equal  to  extreme  (equal,  moderately  more,  strongly  more, 
very  strongly  more,  extremely  more) .  These  values  are  the 
result  of  comparing  the  more  preferred  alternative  to  the 
lesser  preferred  alternative.  When  comparing  a  lesser 
preferred  alternative  to  a  more  preferred  alternative,  as 
expected,  the  numerical  values  would  be  the  reciprocals  of  the 

above,  or  The  aspect  of  judgements  is 
discussed  in  the  next  section. 

C.  PAIRWISE  COMPARISON  SCALING 

A  primary  concern  of  AHP  is  with  scaling.  Saaty  states 
"The  Analytic  Hierarchy  Process  is  rigorously  concerned  with 
the  scaling  problem  and  what  sort  of  numbers  to  use,  and  how 
to  correctly  combine  the  priorities  resulting  from  them"  [Ref. 
1 : p .  10]  .  The  table  Saaty  devised  to  use  when  making  pairwise 
comparisons  is  shown  in  Figure  1  [Ref.  l:p.  15]. 
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Intensity  of  im¬ 
portance  on  an 
absolute  scale 


2,4, 6, 8 


Reciprocals 


Rationals 


Definition 

Explanation 

Equal  importance 

Two  activities 
contribute 
equally  to  the 
objective 

Moderate 

importance  of  one 
over  another 

Experience  and 
judgement 
strongly  (sic) 
favor  one 
activity  over 
another 

Essential  or 
strong  importance 

Experience  and 
judgement 
strongly  favor 
one  activity  over 
another 

Very  strong 
importance 

An  activity  is 
strongly  favored 
and  its  dominance 
demonstrated  in 
practice 

Extreme  importance 

The  evidence 
favoring  one 
activity  over 
another  is  of  the 
highest  possible 
order  of 
affirmation 

Intermediate 
values  between  the 
two  adjacent 
judgements 

When  compromise 
is  needed 

If  activity  i  has  one  of  the  above 
numbers  assigned  to  it  when  compared 
with  activity  j,  then  j  has  the 
reciprocal  value  when  compared  with  i 

Ratios  arising 
from  the  scale 

If  consistency 
were  forced  by 
obtaining  n  num¬ 
erical  values  to 
span  the  matrix 

Figure  1.  The  AHP  Point  Scale  for  Pairwise  Comparisons. 
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D.  PAIRWISE  COMPARISONS 


The  aspect  of  pairwise  comparisons  that  is  described  below 
assumes  a  decision  maker  is  evaluating  a  problem  of  m 
attributes  and  n  alternatives  using  AHP. 

The  attributes  that  contribute  to  the  value  of  each 
alternative  to  be  evaluated  (Saaty  refers  to  the  attributes  as 
"criteria"),  are  evaluated  by  pairwise  comparison  by  the 
decision  maker.  In  an  assessment  of  m  attributes,  the 

decision  maker  is  required  to  perform  - 1  ” 1  pairwise 

comparisons.  The  decision  maker  would  construct  a  pairwise 
comparison  matrix  with  m  rows  and  m  columns  where  each  row  and 
column  represent  an  attribute.  The  number  of  pairwise 
comparisons  is  reduced  from  m2  by  two  requirements  of  AHP. 
First,  the  matrix  will  be  reciprocal.  That  is  each  element 
rij  of  the  pairwise  comparison  matrix  is  the  reciprocal  of 

rji  ,  ^ 


rtj  = -  Vi, j=l,  .  .  .  ,m. 

Zji 

Second,  main  diagonal  elements  of  the  matrix  will  always  have 
entries  of  one.  That  is 
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zii  =  1  Vi  =  1 ,  .  .  .  ,  m. 

The  pairwise  comparison  procedure  is  performed  at  every 
level  of  the  hierarchy,  with  the  exception  of  the  alternatives 
themselves,  the  lowest  level  of  the  hierarchy.  In  other 
words,  if  there  were  one  or  more  levels  of  subcriteria, 
pairwise  comparisons  in  the  manner  described  above  would  be 
performed. 

In  the  final  or  lowest  level  of  the  hierarchy,  the 
decision  maker  would  make  pairwise  comparisons  of  alternatives 
one  attribute  at  a  time.  The  decision  maker  would  evaluate 
each  alternative  by  attribute.  Ultimately,  the  decision  maker 
will  have  m  matrices  (one  for  each  attribute)  of  size  nxn. 

Each  matrix  is  formed  by  making  n ( 1  *  pairwise  comparisons  of 
the  alternatives. 

The  pairwise  comparison  matrices  are  said  to  be  consistent 
if  there  is  a  vector  w  of  size  n,  in  the  case  of  alternatives 
(u  would  be  of  size  m  in  the  case  of  attributes) ,  such  that 

<l)  • 

rH  =  — -  Vi  ,  j  =  1 ,  .  .  .  ,  n.  (l) 

Otherwise,  the  matrix  is  not  consistent.  Note  that  these 
equations  imply  that 
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zijmzik'zkj>  Vi'j.k  (2) 

for  consistency.  The  vector  u  is  made  unique  by  normalizing 
by  dividing  by  its  sum.  Thus, 

n 

£  Uj-l. 
i- 1 

If  we  refer  to  the  matrix  of  pairwise  comparisons  as  R;  R 
is  consistent  if,  and  only  if 

Rtt  =  nu . 

In  a  problem  where  there  is  some  inconsistency  present,  AHP 
solves 

where  X ^  is  the  principal  eigenvalue  of  R.  This  leads  to  an 

approximation  of  u  whose  entries  correspond  to  the  weights  of 
the  alternatives  or  attributes.  To  determine  the  amount  of 
inconsistency  and  determine  if  the  amount  of  consistency  is 
acceptable,  Saaty  developed  the  consistency  index  (Cl)  defined 
as 

CJ= 

(n-1 ) 

Saaty  recommends  if  Cl  £  0.1,  accept  the  estimate  of  ». 
Otherwise,  attempt  to  improve  consistency.  He  does  not 
specifically  state  what  course  of  action  the  decision  maker 
should  undertake  to  improve  consistency  [Ref.  l:p.  13]. 
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The  decision  maker  could  be  forced  into  consistency  by 
making  just  n  pairwise  comparisons  in  the  first  row  of  the 
pairwise  comparison  matrix.  The  first  pairwise  comparison 
would  be  alternative  one  compared  to  itself,  which  is  by 
definition  unity.  Excluding  this  comparison,  (n-1)  pairwise 
comparisons  are  all  that  are  necessary.  In  this  way,  the 
decision  maker  would  obtain  the  entries  for  the  first  row  of 
the  pairwise  comparison  matrix,  and  define  the  weights  based 
on  those  entries 

(ril  '  ri2'  •  •  •  «  *i„>  5  («l#  w2,  .  .  .  ,  o . 

The  entries  for  the  rest  of  the  matrix  could  be  obtained  by 
defining 


r  =  — 


V  i,  ;'  =  1,  .  .  .  ,fl 


and  again  normalizing  weights  to  sum  to  one.  With  these 
weights  obtained  exclusively  from  the  first  row  the  pairwise 
comparison  matrix,  every  element  of  the  matrix  could  be 
obtained.  The  resulting  matrix  R  would  be  perfectly 
consistent . 

However,  AHP,  to  its  credit,  does  not  force  this 
consistency  on  the  decision  maker.  By  requiring  n(”~1) 

comparisons,  AHP  makes  the  vector  «  over-determined  and  allows 
inconsistencies . 
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Therefore,  the  pairwise  comparison  matrix  could  very 
likely  contain  inconsistencies.  With  the  presence  of 
inconsistencies,  there  is  no  exact  solution  for  the  vector  &> 
such  that  Equation  (1)  holds  for  every  i  and  j.  The  question 
is  how  to  find  an  «  that  "best"  fits  these  equations  when 
inconsistency  is  present.  With  little  justification,  AHP  uses 
the  eigenvalue  method  previously  described.  The  advantages  of 
this  method  are  (i)  if  the  pairwise  comparison  matrix  is 
consistent,  ^max  =  -n'  and  ( i  i )  it  allows  evaluation  of 

consistency  by  the  consistency  index  (Cl)  defined  above. 

In  Chapter  IV  we  discuss  and  evaluate  other  methods  of 
determining  the  vector  u.  All  methods  produce  exact  results 
if  the  decision  maker  is  consistent.  We  evaluate 
approximations  by  defining  the  error  between  the  pairwise 
comparison  matrix  and  one  produced  from  the  vector  of  weights, 
«. 


E .  EXAMPLE 

In  order  to  illustrate  how  to  use  AHP  in  a  problem, 
attached  as  Appendix  A  to  this  thesis  is  the  example  of 
choosing  the  best  house  to  buy  using  AHP  from  Saaty's  article 
cited  above  [Ref.  1] . 

F.  INSIGHTS 

In  light  of  this  chapter's  discussion  concerning  AHP, 
several  questions  may  have  occurred  to  the  reader.  For 
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example,  given  the  AHP  system  of  pairwise  comparison  scaling, 
how  does  the  decision  maker  rate  three  alternatives  if  the 
first  is  judged  to  be  of  extreme  importance  to  the  second,  and 
the  second  is  determined  to  be  of  extreme  importance  to  the 
third?  There  is  no  81  on  the  scale,  which  is  what  is  required 
of  r13  if  -t12=r23=9  and  Equations  (l)  and  (2)  have  to  hold. 

Additionally,  how  does  the  restriction  to  an  integer  scale 
compromise  the  accuracy  of  pairwise  comparisons?  Consider 
three  alternatives  which  result  in  the  following  pairwise 
comparisons.  The  first  is  preferred  by  a  factor  of  three  to 
the  second  (r12  =  3)  .  The  third  is  preferred  by  a  factor  of  two 

to  the  second  (r32=2) .  By  how  much  is  the  first  alternative 
preferred  to  the  third?  Equation  (2)  shows  that  for 
consistency  to  hold,  r13  =  1.5,  but  that  is  not  on  the  AHP  scale 

and  so  could  not  have  been  chosen. 

These  problems  and  others  will  be  discussed  thoroughly  in 
the  next  chapter. 
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III.  CRITIQUE  OF  THE  ANALYTIC  HIERARCHY  PROCESS 


A.  BACKGROUND 

The  Analytic  Hierarchy  Process  contains  several  flaws, 
three  of  which  are  discussed  in  this  chapter.  Two  issues, 
scaling  and  ordering,  are  significant  concerns,  yet  the 
problem  of  rank  reversal  represents  a  flaw  in  methodology  and 
is  considered  the  most  significant  issue  regarding  the 
usefulness  of  AHP.  It  is  for  this  reason  that  the  problem  of 
rank  reversal  must  be  viewed  as  far  more  significant  than  any 
of  the  other  problems.  The  result  is  that  AHP  can  produce 
rank  orderings  that  are  not  consistent  with  the  underlying 
preferences  of  the  decision  maker.  In  Chapter  IV,  we  show  how 
to  avoid  this  while  maintaining  the  ratio  scale. 

1 .  Rank  Reversal 

By  adding  or  deleting  an  alternative,  reversal  in  the 
ranking  of  the  other  alternatives  can  occur,  even  though  this 
addition  or  deletion  causes  no  change  in  the  decision  maker's 
pairwise  comparisons.  As  ironic  as  it  may  seem,  AHP  can  tell 
a  decision  maker  that  his  or  her  preference  of  alternatives 
would  change  if  an  additional  alternative  were  to  be 
considered.  Even  if  the  new  item  considered  has  the  same 
attribute  levels  as  an  original  alternative,  AHP  could  tell 
the  decision  maker  to  change  original  preferences.  This  is 
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what  is  described  as  rank  reversal.  The  concept  will  be 
covered  in  greater  depth  later  in  this  chapter.  This  has  been 
much  discussed  in  the  literature.  See,  for  example,  Schoner 
and  Wedley  [Ref.  2],  Howard  [Ref.  3],  Holder  [Ref.  4],  Roper 
and  Sharp  [Ref.  5]  and  Belton  and  Gear  [Ref.  6]  .  Saaty 
maintains  that  rank  reversals  are  justified  because  one  rates 
the  uniqueness  of  an  alternative.  The  presence  of  a  duplicate 
of  this  alternative  must,  he  claims,  decrease  the  value  of  the 
original  alternative,  even  if  the  decision  maker  says  it  does 
not  affect  his  or  her  relative  strengths  of  preferences. 

2.  Pairwise  Comparison  Scaling 

Pairwise  comparisons  are  restricted  to  a  nine  point 
integer  scale.  This  necessarily  leads  to  inconsistency  since 
no  attribute  or  alternative  can  be  1.5,  4.5  or  10  times  more 
important  than  another. 

3 .  Weight  Pitting 

The  most  appropriate  way  to  determine  the  weights  wi 

from  pairwise  comparisons  may  not  be  the  eigenvector  method 
used  by  AHP.  In  an  article  by  Hihn  and  Johnson  [Ref.  7] ,  16 
methods  are  studied,  and  there  appears  to  be  no  reason  to 
believe  AHP  generates  the  most  desirable  solutions  using  the 
eigenvector  technique. 

Now  we  move  to  a  discussion  of  how  AHP  is  used  in 
practice  and  how  AHP  could  be  changed  in  a  fundamental  way. 
In  a  problem  in  which  AHP  is  used,  let  us  assume  that  there 
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are  n  attributes  indexed  i=l,2,...,n  with  the  weight  of 
attribute  i  given  the  value  .  The  attributes  will  be 

normalized  so  that 

n 

E  Q  j  =  1  . 

i- 1 

We  would  like  to  determine  a  vector  of  weights 
»  =  (u1(  o2,  .  .  .  ,  <on)  7  that  denotes  positive  weights  obtained  by 

the  pairwise  comparisons.  We  will  define 


rij  = 


This  represents  the  ratio  of  the  weights  of  attributes  i  and 
j  which,  in  Saaty's  method,  is  determined  through  pairwise 
comparisons.  As  was  pointed  out  in  Equation  (2),  if  the 
decision  maker  is  consistent,  the  ratios  will  satisfy, 


rii  = 


a>i 


0)i  <ok 

—  • — -  =rikrkj 
U)k  u  J 


V  i  ,  j  ,  k. 


If  there  is  inconsistency,  determining  the  "best"  w  is  a 
nontrivial  matter.  There  are  n  w_i '  s  to  be  found  using 

rij7-'  s,  so  the  problem  is  how  to  best  fit  the  u/s  from 

an  over-determined  data  set . 

In  AHP,  the  matrix  of  these  ratios,  or  comparisons, 
will  approximate  the  vector  u  of  the  weights  by  solving  the 
following  equation 
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^11  *12  •  •  *  ri n 

<*>1 

X2\  I22  •  *  '  I2n 

fc>2 

-  1 

<*>2 

Amax 

Znl  rn2  •  •  •  rnn 

<*>« 

“n 

(3) 


where  the  matrix  of  ratios  (data  from  pairwise  comparisons) 
will  be  labelled  R.  Equation  (3)  can  be  expressed  in  matrix 
notation  as  shown  below: 


Characteristics  of  matrix  R  include 

r,,  =  — —  matrix  is  reciprocal, 

’ 

rn  =  l. 

The  diagonal  entries  of  R  are  all  one,  since  a  pairwise 
comparison  of  an  attribute  with  itself  must  be  one. 

Saaty' s  method  forms  the  matrix  R  by  performing 

n(n2~1)  pairwise  comparisons.  The  next  step  is  to  determine 

the  largest  eigenvector  of  R  and  the  corresponding  right 
eigenvector.  This  vector  is  normalized  to  give  the  relative 
weights.  Let  represent  the  largest  eigenvalue  of  R.  We 

list  the  following  results  found  in  the  AHP  literature: 

•  R  is  reciprocal, 

•  R  is  not  necessarily  consistent, 
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•  R  is  consistent  if  and  only  if  ^■mAX  =  n, 

®  ^max  ^  ^  • 

In  AHP,  the  nine  point  scale,  as  previously  described,  is  used 
to  make  comparisons.  Saaty  advises  if 

v  n 

—222 -  *0.1  , 

n 

use  the  results  obtained,  since  comparisons  are  "consistent 
enough."  However,  if 

1  v  -Ti 

— 225 >0.1, 

n 

identify  the  inconsistencies,  modify  the  comparisons  and 
recalculate.  This  appears  to  be  a  rule-of-thumb  with  no 
theoretical  foundation. 

B.  RANK  REVERSALS 

An  axiom  of  rational  decision  making  implies  that  addition 
or  deletion  of  a  new  alternative,  with  no  changes  in  relative 
preferences  for  existing  alternatives ,  should  never  cause  a 
change  in  the  existing  ranking  when  ranking  alternatives. 
Saaty  argues  "rank  reversals  do  occur  in  practice  in  a  way 
that  does  not  satisfy  these  assumptions"  [Ref.  8:p.  13].  In 
way  of  an  explanation,  Saaty  continues,  "On  seeing  too  many 
copies  of  an  attractive  alternative,  one  abandons  it  for  a 
less  attractive  one  that  is  unique"  [Ref.  8:p.  13].  In  an 
example  of  rank  reversal  later  in  this  section,  rank  reversal 
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of  three  tanks  occurs  when  a  duplicate  of  one  of  the  tanks  is 
added.  Presumably,  the  decision  maker  is  not  planning  on 
fighting  a  war  with  one  tank  and  is  aware  that  there  are  other 
tanks  in  existence  of  all  types  of  tanks  considered.  In  fact, 
it  could  be  argued  that  in  this  case  uniqueness  of  an 
alternative  would  be  a  weakness.  In  this  case,  however,  AHP 
rates  an  alternative  as  less  preferable  when  a  duplicate  of 
that  alternative  is  considered  despite  the  fact  that 
uniqueness  is  clearly  not  an  advantage. 

Saaty  shows  the  frequency  of  rank  preservation  of  all 
alternatives  for  1000  random  cases  involving  two  to  nine 
criteria  and  two  to  nine  alternatives  in  distributive,  ideal 
and  utility  modes  of  AHP  [Ref.  8] .  In  the  distributive  mode, 
weights  derived  from  paired  comparisons  are  normalized  to  add 
to  1.0.  The  ideal  mode  divides  by  the  weight  of  the  highest 
ranked  alternative  for  each  criterion  making  the  largest 
weight  to  be  1.0.  The  utility  mode  is  a  transformation  of  the 
ideal  mode  to  an  interval  scale.  The  trend  in  all  three  cases 
shows  the  rate  of  rank  preservation  decreases  as  number  of 
criteria  and  alternatives  increase.  In  other  words,  the  more 
complex  the  problem  in  terms  of  criteria  and  alternatives ,  the 
more  likely  it  is  that  one  will  have  rank  reversal.  In  the 
distributive  mode  with  two  criteria  and  two  alternatives, 
there  were  963  c^ses  out  of  1000  of  rank  preservation.  When 
nine  criteria  and  nine  alternatives  were  used,  rank  was 
preserved  just  526  times  out  of  the  1000  cases.  Similarly  for 
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the  ideal  mode,  967  cases  of  rank  preservation  for  two 
criteria  and  two  alternatives  reduced  to  832  cases  with  nine 
alternatives  and  nine  criteria.  Finally  for  the  utility  mode, 
903  cases  of  rank  preservation  with  two  criteria  and  two 
alternatives  were  reduced  to  just  664  cases  of  rank 
preservation  when  nine  criteria  and  nine  alternatives  were 
used  [Ref.  8]. 

1.  Duplication 

In  1983,  shortly  after  the  introduction  of  AHP,  it  was 
shown  that  the  introduction  of  a  duplicate  alternative  could, 
in  some  cases,  cause  a  rank  reversal  of  the  other  alternatives 
[Ref.  6 ]  . 1  Saaty  responded  to  this  by  arguing  that  rank 
reversals  are  justified  because  an  alternative  loses  some  of 
its  appeal  if  it  is  not  unique.  In  reality,  sometimes  a 
duplicate  may  reduce  the  value  of  an  alternative,  but  in  AHP, 
duplicates  of  alternatives  always  reduce  the  value  of  the 
original  alternatives.  The  decision  maker  has  no  choice.  If 
a  duplicate  alternative  is  introduced  into  the  decision 
problem,  the  value  of  the  original  alternative  is  reduced. 
The  rank  reversal  will  still  occur  even  if  the  uniqueness  of 
an  alternative  is  irrelevant  to  the  decision  maker. 

This  problem  is  caused  by  the  particular  normalizing 
of  weights  in  AHP.  This  is  inherent  to  AHP  and  cannot  be 


'The  rank  reversal  example  of  Belton  and  Gear  is  reproduced  by 
Schoner  and  Wedley  in  Reference  2. 
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eliminated  without  changing  the  way  AHP  normalizes  weights, 
which  is  exactly  what  we  hope  the  distributive  method  has 
done,  but  the  article  [Ref.  8]  fails  to  explain  the  source  of 
the  rank  reversals.  The  474  cases  of  rank  reversal  out  of  the 
1000  cases  in  which  nine  criteria  and  nine  alternatives  were 
used  with  the  distributive  method  could  be  due  to 
inconsistency  in  pairwise  comparisons,  or  renormalizing 
weights  without  regard  to  units  of  measurement .  We  can  not  be 
sure  if  the  distributive  method  is  a  viable  alternative  to 
AHP.  There  can  be  two  sources  of  rank  reversal.  The  first 
source  is  inconsistency  in  pairwise  comparisons  by  the 
decision  maker.  The  second  source  is  from  normalizing 
attributes  of  alternatives  in  a  way  that  does  not  maintain  a 
ratio  scale.  The  former  source  of  rank  reversal  does  not 
appear  tc  have  a  solution.  The  latter  source  can  be  solved  by 
either  of  the  modifications  described  in  this  thesis.  We  must 
caution  that  Saaty's  distributive  method  might  cause  many  or 
all  of  the  rank  reversals  that  occurred  by  a  flawed  method  of 
normalizing. 

2 .  Near  Copies 

In  response  to  rank  reversals  upon  the  introduction  of 
duplicate  alternatives,  Saaty,  rather  than  recognize  this 
issue  as  a  potential  source  of  error  in  the  methodology  of 
AHP,  suggested  elimination  of  alternatives  from  consideration 
that  score  within  10  percent  of  another  alternative.  In  an 


21 


example  later  in  this  chapter,  the  additional  alternative 
introduced,  the  third  tank,  is  not  within  10  percent  of  either 
of  the  first  two  tanks  in  either  of  the  two  attributes 
evaluated,  yet  a  rank  reversal  still  occurs. 

3.  Rank  Reversal  Example 

What  follows  are  examples  of  how  the  Analytic 
Hierarchy  Process  (AHP)  can  produce  a  rank  reversal  in 
alternatives  when  an  additional  alternative  is  added.  It  also 
follows  that  a  rank  reversal  can  occur  when  an  alternative  is 
eliminated  from  consideration.  These  examples  simulate  making 
a  decision  on  selecting  or  valuing  a  weapon  system,  in  this 
case  a  tank.  However,  much  like  the  example  of  using  AHP  to 
compare  houses  (which  is  in  Appendix  A  to  this  thesis) ,  this 
example  is  intended  to  be  generic  and  equally  applicable  to 
all  ranking  or  valuing  problems.  Therefore,  the  problem  of 
rank  reversal  is  not  a  special  case  or  exception  when  AHP  is 
used. 

a.  Attributes 

Assume  that  there  are  two  attributes,  survivability 
and  firepower.  Further  assume  that  you  decide  that  firepower 
is  1.25  times  as  important  as  survivability.2  Weights  are 

(  -• ,  )  for  survivability  and  firepower,  respectively.  This 


Although  the  literature  on  AHP  calls  for  using  the  scale  in 
Figure  1,  the  computer  application  of  AHP,  "Expert  Choice",  does 
allow  for  a  continuous  set  of  comparison  numbers  to  be  used. 
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maintains  the  weight  priorities  (1.25x-±=-|)  ,  and  meets  the  AHP 

requirement  that  weights  sum  to  one.  The  example  of  how  these 
attributes  satisfy  the  equality  of  Equation  (3)  is  shown  below 
where  XmMX=n  =  2 


S  F 
S 

F 

The  labels  S  and  F  represent  survivability  and  firepower, 
respectively.  The  matrix  is  consistent,  as  it  must  be  in  any 
2x2  reciprocal  matrix  with  entries  of  one  on  the  main 
diagonal . 

(1)  Survivability.  On  comparing  three  types  of 
armor,  reactive  armor  is  preferred  1.5  over  applique,  and 
applique  is  preferred  2  over  rolled  homogeneous  armor. 

(2)  Firepower.  A  tank  with  a  130mm  main  gun  is 
preferred  1.4  over  a  tank  with  a  120mm  main  gun.  A  120mm  main 
gun  is  preferred  1.5  over  a  105mm  main  gun.  A  105mm  main  gun 
is  preferred  2.0  over  a  90mm  main  gun. 

b.  Comparing  two  Tanka  using  AHP 

Tank  1  has  applique  armor  and  a  120mm  main  gun. 
Tank  2  has  reactive  armor  and  a  105mm  main  gun.  So  Tank  1  has 
more  firepower  but  is  less  survivable  than  Tank  2. 
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The  consistent  matrix  equation  satisfying  Equation 
(3)  from  the  pairwise  comparisons  of  Tank  1  with  Tank  2  for 
the  attribute  of  survivability  is  shown  below: 


T1 

T2 


T1  T2 


1  2 

4 

4 

3 

10 

=  2 

10 

1  i 

6 

6 

2  1 

10 

10 

Similarly,  the  matrix  equation  satisfying  Equation  (3)  from 
the  pairwise  comparisons  of  Tank  1  with  Tank  2  for  the 
attribute  of  firepower  is  shown  below: 


T1 

T2 


T1  T2 


1 

3 

6 

6 

2 

10 

=  2 

10 

2 

4 

4 

3 

X 

10 

10 

Figure  2  shows  the  method  used  to  determine 
relative  value  of  alternatives  once  the  weights  of  attributes 
have  been  assigned  and  pairwise  comparisons  of  alternatives 
have  been  made. 


ATTRIBUTES 

RELATIVE 

VALUE 

TANK 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

0.4 

0.6 

0.511 

TANK  2 

0.6 

0.4 

0.489  | 

Flgure'"^^^Con5arTson,l,^l^Tw^Tankl^asTn^AH^ 
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The  figure  shows  Tank  1  is  preferred  to  Tank  2,  or 


Tank  1  >  Tank  2 . 

If  we  let  Rar  be  the  pairwise  comparison  matrix 

obtained  by  pairwise  comparisons  of  the  relative  value  of  the 
tanks,  we  obtain  the  following  matrix: 

T1  T2 
1  1.045 

0.957  1 


T1 

R2P  = 
ar  T2 


c.  Additional  Alternative 

Tank  1  has  applique  armor  and  a  120mm  main  gun,  and 
Tank  2  has  reactive  armor  and  a  105mm  main  gun.  These  are  the 
same  tanks  as  in  the  section  above.  Tank  3  has  rolled 
homogeneous  armor  and  a  130mm  main  gun.  Tank  2  has  the  most 
survivability  and  Tank  3  has  the  most  firepower.  Tank  3  has 
the  least  survivability  and  Tank  2  has  the  least  firepower. 
Tank  1  was  rated  between  Tanks  2  and  3  in  both  attributes. 


ATTRIBUTES 

RELATIVE  1 

VALUE 

TANK 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

1  TANK  1 

0.333 

0.326 

0.329 

1  TANK  2 

0.500 

0.217 

0.343 

I  TANK  3 

0.167 

0.457 

0.328 

Figure  ?^TomparTsorT*o^^iree  tanlcs^osTn^AHI^ 
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When  Tank  3  is  included,  Tank  2  becomes  preferred 
to  Tank  1  which  is  in  turn  preferred  to  Tank  3 .  Figure  3 
shows  Tank  2  is  preferred  to  Tank  1,  which  is  preferred  to 
Tank  3,  or 


Tank  2  >  Tank  1  >  Tank  3 . 

Note  that  this  reversal  happened  even  though  no  changes  were 
made  to  the  relative  importance  of  the  options  in  either 
survivability  or  firepower,  or  between  these  two  attributes. 
If  we  let  R3T  be  the  pairwise  comparison  matrix  obtained  by 

pairwise  comparisons  of  the  relative  values  of  the  three 
tanks,  we  obtain  the  following  matrix: 


T1 

T2 

T3 

T1 

1 

0.959 

1.003 

T2 

1.043 

1 

1.046 

T3 

0.997 

0.956 

1 

If  the  ratio  scale  had  been  maintained,  the  pairwise 
comparisons  between  Tank  1  and  Tank  2  should  be  the  same  as 
they  were  in  J?ajP.  This  is  not  the  case. 

d.  Explanation  of  Rank  Reversal 

In  the  original  formulation  that  compared  two 
tanks,  the  dilution  of  components  of  relative  value  can  be 
seen  in  a  comparison  of  the  two  tables  showing  component 
relative  values  before  and  after  Tank  3  was  included. 
Originally,  the  component  in  the  table  contributing  the 
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largest  amount  to  either  tank's  relative  value  is  represented 
by  the  firepower  attribute  of  Tank  1.  This  can  be  read  off 
the  table  as  0.333.  This  is  significantly  higher  than  the 
relative  value  component  of  survivability  of  Tank  2  which  is 
only  0.2666.  With  the  addition  of  Tank  3  which  has  a  large 
rating  for  firepower,  the  relative  value  components  of 
firepower  for  the  two  original  tanks  (Tank  1  and  Tank  2)  are 
significantly  reduced.  Once  Tank  3  is  added  as  an 
alternative,  all  components  of  relative  attributes  of  the 
original  two  tanks  are  reduced.  In  this  case,  Tank  3  was 
rated  higher  in  firepower  than  survivability.  Therefore,  the 
firepower  rating  of  Tank  1  (originally  0.333  then  0.181  for  a 
reduction  of  0.152)  had  a  greater  reduction  in  magnitude  than 
the  survivability  rating  of  Tank  2  (originally  0.2666  then 
0.222  for  a  reduction  of  only  0.044  6)  .  The  greater  reduction 
in  magnitude  of  the  firepower  attribute  of  Tank  1  led  to  the 
rank  reversal  with  Tank  2 . 

e.  Duplication  of  Alternative 

Now  let  us  suppose  a  fourth  tank  is  to  be 
considered.  The  fourth  tank  has  the  same  characteristics  as 
Tank  2.  Therefore,  since  it  also  has  reactive  armor,  it  will 
be  preferred  by  3  to  1  over  Tank  3  in  survivability,  which  has 
rolled  homogeneous  armor.  It  is  preferred  by  1.5  to  1  over 
Tank  1  with  survivability  characteristic  of  applique  type 
armor.  Similarly  in  firepower,  its  small  diameter  main  gun  is 
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least  preferred.  Both  attributes  must  be  renormalized  and 
multiplied  by  the  weight  of  the  attribute  to  obtain  the  new 
relative  value  of  the  tanks.  This  is  done  in  Figure  4. 


I 

ATTRIBUTES 

RELATIVE 

VALUE 

TANK 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

0.2222 

0.2679 

0.2475 

TANK  2 

0.3333 

0.1783 

0.2472 

TANK  3 

0.1112 

0.3755 

0.2581 

TANK  4 

0.3333 

0 . 1783 

0.2472 

Figure  4.  Comparison  of  four  tanks  using  AHP. 


The  decision  maker  might  consider  the  uniqueness  of 
a  tank  to  be  irrelevant,  but  nevertheless,  AHP  produces  a 
dramatic  rank  reversal  in  this  case.  Recall  that  originally 
AHP  produced  a  preference  order  of  two  tanks  as  Tank  1  >  Tank 

2.  With  three  tanks,  the  order  becomes  Tank  2  >  Tank  1  >  Tank 

3 .  Now  by  adding  a  fourth  tank  with  the  same  characteristics 
as  Tank  2,  the  preference  of  the  four  tanks  is  3,  1,  2,  4, 
that  is  Tank  3  >  Tank  1  >  Tank  2  *  Tank  4  (the  "worst"  becomes 
the  "best" ! ) . 

f.  Discussion  of  Results 

The  reversal  in  preference  order  of  the  first  two 
tanks  by  the  addition  of  a  third  is  caused  by  flawed  logic  in 
AHP  and  the  disregard  for  measurement  units.  Even  though 
there  are  no  natural  units  for  measuring  survivability  and 
firepower,  one  must  define  an  underlying  scale  based  on  a 
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standard.  Both  the  scale  intervals  and  standard  can  be 
arbitrarily  chosen,  but  once  they  are,  measurements  must  be 
made  consistent  with  this  scale. 

C.  PAIRWISE  COMPARISON  SCALING 

There  is  inherent  inconsistency  in  the  nine  point  scale 
used  by  AHP.  The  nine  point  pairwise  comparison  scale  is 
defined  in  Figure  1  in  Chapter  II. 

Immediately,  it  is  possible  to  pose  theoretical  problems 
impossible  to  formulate  when  restricted  to  this  nine  point 
scale.  This  is  the  case  if  one  is  faced  with  three 
alternatives,  A,  B,  and  C  with  the  following  pairwise 
comparison  results.  Alternative  C  is  deemed  to  have  extreme 
importance  over  B,  and  B  is  deemed  to  have  extreme  importance 
over  A.  One  would  assume  C  would  have  much  more  than  extreme 
importance  over  A,  but  this  is  not  possible  with  AHP.  To  say 
that  alternative  C  is  just  of  extreme  importance  over  A  defies 
logic.  That  would  imply  that  the  decision  maker  should  be 
indifferent  between  B  and  A  (after  all,  C  is  of  extreme 
importance  over  both  these  alternatives)  .  Perhaps  the 
decision  maker  should  now  decide  alternative  C  now  only  has 
strong  importance  to  B  and  B  only  has  strong  importance  to  A 
so  C  can  have  extreme  importance  over  A.  This  new  hierarchy 
is  neither  consistent,  nor  does  it  explain  why  a  decision 
maker  should  revise  his  or  her  estimate  of  the  pairwise 
comparison  of  alternatives. 
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The  situation  above  was  entirely  theoretical  with  generic 
alternatives.  But  what  if  the  problem  was  travel  from  the 
west  coast  to  the  east  coast,  and  alternative  A  was  walking, 
alternative  B  was  driving  a  car  and  alternative  C  was  flying 
a  plane.  It  seems  obvious  alternative  C  would  be  preferred  by 
more  than  a  factor  of  nine  to  alternative  A. 

Inconsistency  is  not  the  exception  with  this  nine  point 
scale.  Almost  any  time  there  are  more  than  two  or  three 
comparisons  to  be  made,  the  scale  will  almost  always  lead  to 
inconsistencies.  To  maintain  consistency,  the  pairwise 
comparisons  of  alternatives  should  be  the  product  of  the  scale 
number  determined.  Take,  for  example,  four  alternatives.  A, 
B,  C,  and  D,  that  are  deemed  of  moderate  importance  over  their 
preceding  alternative.  Therefore,  alternative  A,  being  the 
least  preferred  alternative,  would  receive  a  1  in  a  pairwise 
comparison  to  itself.  Alternative  B  would  receive  a  3  in 
comparison  to  A  (moderately  preferred) .  Similarly, 
alternative  C  receives  a  score  of  3  compared  to  B,  or  9 .  But 
what  score  does  D  receive?  It  is  moderately  preferred  to  C, 
but  C  already  has  the  highest  score  allowed  on  the  nine  point 
scale . 

In  theory,  there  should  be  no  limit  to  the  scale.  The 
scale  should  be  open  ended,  and  not  restricted  to  a  maximum 
value  of  nine . 

One  simple  way  to  avoid  the  scaling  problems  described 
above  would  by  to  insure  the  decision  maker  stayed  within  the 
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scale.  The  decision  maker  would  identify  the  extreme 
attributes,  that  is  the  most  and  least  preferred  attributes. 
The  least  preferred  attribute  would  be  defined  as  one  on  the 
scale,  and  the  most  preferred  would  be  defined  as  nine.  All 
other  attributes  would,  therefore,  fall  somewhere  in  between 
these  two  extremes  and  all  attributes  would  be  within  the 
scale . 

D.  WEIGHT  FITTING 


In  any  evaluation  of  attributes  that  allows  inconsistency, 
the  user  must  somehow  be  able  to  determine  how  much 
inconsistency  there  is  in  the  evaluation.  Given  a  matrix  R  = 


[r^]  ,  suppose  the  weight; 

expression  that  measures  the 
ratios  is 

n  n 

s  2 

j*i  j*i 


(0  have  been  obtained.  One 
error  between  the  weights  and  the 


A  second  and  more  tractable  error  measure  is 

Error=2  2  (  o.r,, -u, ) 2 .  (4) 

i=i  j= i 

Both  of  these  expressions  are  zero  if,  and  only  if,  R  is 
consistent.  One  definition  of  the  "best"  weights  w  is  that 
they  minimize  the  expression  in  Equation  (4)  for  a  given 
matrix  R.  This  is  the  method  of  least  squared  error  (LSE)  for 
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evaluation  comparisons,  and  was  the  first  method  we  selected 
to  evaluate  preference  fitting.  Without  further  comment,  we 
point  out  that  the  above  equation  is  not  the  unique  way  to 
evaluate  consistency.  In  fact,  the  article  by  Hihn  and 
Johnson  [Ref.  7]  lists  and  describes  16  methods  of  finding  the 
"best"  <i>  for  a  given  R  for  various  definitions  of  "best".  It 
appears  that  none  of  the  methods  differ  from  each  other  a 
great  deal,  but  Saaty's  method  was  one  of  the  poorest. 

Saaty'  method,  the  second  method  we  evaluated  by  using 
preference  fitting,  is  to  solve 

where  Amax  represents  the  largest  eigenvalue  of  the  matrix  R. 
In  all  cases,  Amax  a  n,  and  when  R  is  consistent,  Xmax  =  n. 

Hihn  and  Johnson  point  out  that  Saaty's  justification  for 
selecting  this  particular  method,  the  right  eigenvector 
technique,  is  that  if  R  is  consistent,  this  technique  produces 
the  solution  A.max  =  n.  But  they  also  point  out  that  all  of  the 

16  methods  discussed  in  Reference  7  produce  this  exact 
solution  in  the  case  of  consistency.  Therefore,  the  only 
method  of  differentiating  the  16  techniques  is  to  evaluate 
each  technique's  results  when  inconsistency  is  present.  Hihn 
and  Johnson  show  that  Saaty's  technique  is  one  of  the  poorest 
of  the  16  techniques  evaluated. 
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The  third  and  final  method  we  studied  in  the  area  of 
preference  fitting  we  call  the  dimension  method.  In  this 
method,  we  solve  the  following  equation 

Ru  =  nu . 

1 .  Approach 

In  each  of  the  three  methods,  the  error  matrix  was 
evaluated  based  on  the  LSE  formula.  This  gave  a  value  for  the 
total  error  of  the  method  used.  The  value  of  this  total  error 
does  not  necessarily  prove  any  one  method  is  better  than 
another.  We  did,  however,  obtain  interesting  results  using 
Saaty's  pairwise  comparison  matrices  from  his  example  of  using 
AHP  to  buy  a  house  [Ref.  1] . 

2 .  Results 

A  3x3  matrix  of  attributes  was  evaluated  using  Saaty's 
right  eigenvector  method,  the  LSE  method  and  what  we  call  the 
dimension  method.  The  matrix  is  reproduced  below  with  results 
of  pairwise  comparisons  as  entries.  As  required,  each  main 
diagonal  entry  is  a  one,  and  the  matrix  is  reciprocal . 


R  = 

16  8 

7  1  4 

1  6  8 

0.17  1  4 

-1  -1  1 

8  4 

0.13  0.25  1 

Using  Saaty's  method,  the  priority  vector  of  weights  was 
determined  as  (  0 .7  54  ,  0 . 181  ,  0 . 06  5  )  T  with  an  error  of  17.82 
(1^=3.136).  We  will  denote  the  matrix  obtained  by  AHP  as 
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Rmaaty •  The  result  using  least  squares  was  a  priority  vector 
of  weights  of  (  0 . 727  ,  0 . 202  ,  0 . 07 1 )  T  and  error  of  7.47.  We 
will  denote  the  matrix  obtained  by  the  LSE  method  as  Rrfr . 
Finally,  the  dimension  method  produced  a  priority  vector  of 
weights  of  ( 0 . 768 ,  0 . 145 ,  0 . 087  ) T  with  an  error  of  12.31.  We 
will  denote  the  matrix  obtained  by  the  dimension  method  as 
Rant-  We  point  this  out  not  as  an  evaluation  of  any 

technique,  but  simply  to  demonstrate  that  different  techniques 
produce  different  results.  Naturally,  the  LSE  method  had  the 
smallest  error.  Next  was  the  dimension  method,  and  finally, 
with  the  highest  error  of  any  of  the  three  methods,  was 
Saaty's  eigenvalue  method.  The  reader  should  compare  the 
following  three  matrices  with  the  one  in  Equation  (5)  and 
decide  which  of  these  is  "closest"  to  R.  Note  that,  as 
expected,  the  LSE  method  fits  the  larger  numbers  in  R  better 
than  the  smaller  ones. 


1 

4 . 16 

11.6 

1 

3.60 

10.24 

&SAATY 

0.24 

1 

2.78 

&LSB  = 

0.28 

1 

2.84 

0 . 09 

0.36 

1 

0.10 

0.35 

1 

R 


DIM 


1  5.29  8.83 

0.28  1  2.84 

0.11  0.6  1 


These  results  were  from  using  these  methods  in  one  of 
the  pairwise  comparison  matrices  for  the  three  alternatives  in 
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just  one  of  the  eight  attributes  considered.  We  were  curious 
to  see  how  each  method  compared  if  used  in  the  overall 
problem.  The  results  of  an  APL  program  used  to  evaluate 
errors  by  method  for  alternative  pairwise  comparison  matrices 
are  in  Appendix  C  to  this  thesis.  Each  of  the  eight  3x3 
matrices  represents  one  of  the  attributes  considered.  Each 
matrix  was  used  to  determine  a  vector  u  of  weights  for  the 
alternatives  and  each  method  was  evaluated  for  the  error  term. 
Naturally,  since  the  criterion  used  is  minimum  least  squares, 
the  LSE  method  had  the  smallest  error.  In  all  cases  except 
one,  the  dimension  method  had  the  next  smallest  error.  In 
seven  of  the  eight  cases,  Saaty's  eigenvalue  method  had  the 
largest  error. 

The  program  in  Appendix  C  also  includes  the  8x8 
pairwise  comparison  matrix  to  approximate  the  vector  u  of 
weights  for  each  attribute,  what  Saaty  refers  to  as  level  one 
of  the  hierarchy.  The  error  term  for  this  matrix  showed  the 
LSE  method  produced  an  error  of  237.75.  Saaty's  eigenvalue 
method  gave  the  next  smallest  error  at  259.06,  and  the 
dimension  method  had  by  far  the  highest  error  term  of  858.40. 
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ZV.  SUGGESTED  MODIFICATIONS  TO  AHP 


A.  DESCRIPTION  OF  MODIFICATION 

The  two  significant  problems  with  AHP  identified  in  the 
previous  chapter,  rank  reversals  and  scaling,  are  addressed  in 
this  chapter.  The  modification  we  recommend  for  AHP  is  one 
that  will  avoid  rank  reversal .  It  can  also  avoid  the  problem 
of  AHP's  nine  point  scale. 

1.  Ratio  Modification 

The  decision  maker,  through  pairwise  comparisons, 
forms  a  matrix  of  ratios  of  pairwise  comparisons.  If  this 
matrix  changes  when  additional  alternatives  are  considered,  of 
course  rank  reversals  can  be  expected.  If  the  decision  maker 
elects  to  reassess  his  or  her  previous  alternatives'  values, 
this  is  a  separate  matter  entirely.  Without  the  decision 
maker  changing  his  or  her  original  preferences,  the  ranking 
(and  matrix  of  ratios)  should  remain  unchanged. 

2.  Scale  Modification 

The  most  extreme  comparison  of  alternatives  should  be 
used  to  establish  the  scaling  for  other  alternatives.  There 
is  nothing  wrong  with  the  original  nine  point  scale,  or  a 
scale  from  1-10,  but  all  numbers  between  one  and  the  scale 
extreme  would  have  to  be  included.  Lacking  this  modification, 
the  scale  should  be  open  ended  to  allow  comparisons  of 
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alternatives  differing  by  orders  of  magnitude  in  value  to  the 
decision  maker. 

Recall  that  AHP  requires  two  types  of  comparisons  and 
normalization  of  weights  assigned.  The  first  comparison 
involves  comparing  the  importance  of  one  attribute  with 
another.  This  phase  of  implementing  AHP  is  unchanged  by  this 
modification.  The  other  type  of  comparison  is  to  determine 
the  weight  of  each  alternative's  characteristics  by  attribute. 
This  phase  of  AHP  should  be  modified  so  that  the  least 
preferred  alternative  in  an  attribute  is  assigned  a  value  of 
one  (or  the  most  preferred) .  By  doing  this,  the  renormalized 
weights  obtained  by  pairwise  comparison  of  attributes  will 
maintain  a  ratio  scale  and  avoid  rank  reversals . 

3.  Weight  Fitting 

There  seems  to  be  no  credible  reason  not  to  use  the 
least  square  error  (LSE)  method  to  fit  weights  to  alternatives 
or  attributes.  The  examples  in  Hihn  and  Johnson  [Ref.  7]  are 
convincing.  The  eigenvalue  method  used  by  AHP  is  mediocre,  at 
best,  among  the  16  techniques  evaluated  for  weight  fitting. 
The  justification  for  use  of  the  eigenvalue  method  in  AHP  is 
that  an  exact  solution  is  obtained  in  the  case  of  consistency. 
But  exact  solutions  are  produced  by  all  16  techniques  in  the 
case  of  consistency. 
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B.  REASON  POR  MODIFICATION 


The  Analytic  Hierarchy  Process  (AHP)  is  currently  used  to 
obtain  coefficients  of  the  optimization  program  for  the  math 
programming  algorithm  used  by  the  Director,  Program  Analysis 
and  Evaluation  to  maximize  future  Army  modernization  subject 
to  a  funding  constraint . 

To  assess  the  desirability  of  a  management  decision 
package  (MDEP) ,  TRAC  decided  to  use  the  AHP  as  a  quantitative 
scale  and  measurement  process.  This  thesis  shows  that  AHP 
will  provide  coefficients  for  the  optimization  algorithm  that 
are  not  measurable  and  can  be  subject  to  rank  reversal  if 
certain  alternatives  are  added  or  deleted  from  the  original 
formulation.  Even  if  rank  reversal  does  not  occur,  how  can 
one  have  any  confidence  in  the  ranking  measures  when  they 
change  due  to  no  changes  in  the  preferences  of  the  user.  The 
idea  of  using  such  a  flawed  system  for  making  major  investment 
decisions  is  very  disturbing.  This  thesis  determines  a  way 
to  include  some  aspects  of  AHP,  but  not  suffer  from  rank 
reversal  problems. 

C.  APPROACH 

Each  MDEP  can  be  converted  into  its  relative  contribution 
to  Future  Army  Moderni;  .  ion  through  multiple  levels  of 
hierarchy.  Although  the  MDEP's  contribution  to  Future  Army 
Modernization  may  be  a  meaningless  value  to  a  decision  maker, 
it  will  be  consistent  in  its  relationship  to  other  MDEPs 
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because  all  MDEPs  will  be  expressed  in  the  same  "units"  of 
Future  Army  Modernization.  Additionally,  if  it  is  necessary 
to  add  new  alternatives,  the  original  alternatives  will 
maintain  the  same  values  of  coefficients  for  the  math 
programming  algorithm. 

This  procedure  is  consistent  for  multiple  level  problems. 
The  alternatives  will  receive  the  same  value  whether  the 
problem  is  solved  with  all  levels  used,  or  if  the  problem  is 
reduced  to  a  single  level  by  cross  multiplying  weights  for 
each  level . 

This  thesis  shows  that  any  model  of  alternatives  and 
attributes  can  be  expanded  to  include  other  attributes  without 
encountering  any  problem  with  rank  reversal . 

1.  Additional  Attributes 

The  weight  of  each  attribute  will  remain  the  same. 
Elements  of  each  attribute  will  not  be  renormalized  to  sum  to 
one  if  additional  alternatives  are  introduced,  or  if  original 
alternatives  are  eliminated.  In  fact,  even  in  the  original 
formulation,  weights  of  alternatives  by  attribute  will  not  be 
normalized  to  sum  to  one.  This  implies  that  the  value  of  each 
original  alternative  will  not  change  because  the  value  of  an 
alternative  is  the  weight  of  each  attribute  times  the  value  of 
the  alternative's  attribute  summed  over  all  attributes.  The 
relative  value  of  alternatives  should  not  change  if  more  or 
less  alternatives  are  considered. 
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In  the  original  formulation,  all  alternatives  by 
attribute  will  be  scaled  by  one  of  two  methods.  First,  the 
decision  maker  can  assign  a  value  of  one  to  the  most  preferred 
alternative  for  one  attribute.  We  call  this  the  "standard 
attribute."  In  other  attributes,  alternatives  are  assigned  a 
value  based  on  the  decision  maker's  value  comparisons  between 
attributes.  The  decision  maker  must  create  a  link  between 
attributes  by  declaring  an  equivalency  between  an  alternative 
of  one  attribute  and  an  alternative  in  other  attributes.  One 
can  not  arbitrarily  decide  that  the  decision  maker  places 
equal  value  on  all  most  preferred  alternatives  in  every 
attribute  and  assign  each  a  value  of  one.  The  most  preferred 
alternative  in  other  attributes  could  be  "better"  (assigned  a 
value  greater  than  one)  or  "worse"  (assigned  a  value  of  less 
than  one)  when  compared  to  the  decision  maker's  "standard 
attribute"  value  of  one.  New  alternatives  are  simply  assigned 
values  based  on  the  original  formulation's  value  in  each 
attribute.  If  the  new  attribute  is  preferred  by  a  factor  of 
three  to  the  "standard  attribute"  from  the  original 
formulation,  it  will  be  assigned  a  value  of  three.  If  the  new 
alternative's  attribute  is  less  preferred  than  the  original 
formulation's  most  preferred  attribute,  the  (0,1)  scale  will 
be  maintained.  Even  in  the  original  formulation,  however,  the 
elements  of  other  attributes  may  not  fit  a  (0,1)  scale. 

When  using  the  second  method,  the  decision  maker  can 
assign  a  value  of  one  to  the  lowest  or  least  preferred 
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alternative  for  one  attribute,  which  we  will  refer  to  as  the 
"standard  attribute."  In  this  attribute,  new  alternatives  are 
simply  assigned  values  based  on  the  original  formulation's 
least  preferred  attribute  values  of  one.  In  other  attributes, 
alternatives  are  assigned  a  value  based  on  the  decision 
maker's  value  comparisons  between  attributes. 

2.  Applicability 

The  modification  to  AHP  discussed  in  this  proposal 
would  be  a  significant  improvement  to  the  method  used  by  PAE 
to  obtain  coefficients  to  the  math  programming  algorithm  to 
optimize  future  Army  modernization.  This  method  would  insure 
no  rank  reversals  in  alternatives  whether  alternatives  are 
duplicated,  eliminated,  or  completely  new  alternatives  are 
added.  In  addition,  the  values  of  each  alternative  will  have 
consistent  units  that  can  allow  comparisons  of  magnitude. 

D.  THE  TANK  PROBLEM  REVISITED 

1.  Description  of  Approaches 

This  section  demonstrates  the  use  of  our  suggested 
modification  to  AHP  in  two  ways.  Both  avoid  rank  reversals 
that  plagued  our  earlier  examples. 

a.  Link  between  Attributes 

To  establish  and  maintain  a  ratio  scale  and  avoid 
rank  reversals,  the  decision  maker  must  compare  the  elements 
of  different  attributes  and  determine  a  relative  value  between 
a  specific  element  of  one  attribute  and  an  element  of  other 
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attributes.  It  is  essential  to  have  a  link  between 
attributes.  Otherwise,  an  alternative  with  an  extremely 
undesirable  attribute  would  give  overwhelming  value  to  other 
alternatives  simply  by  assigning  the  least  preferred 
alternative  a  value  of  one.  We  assume  the  decision  maker, 

given  the  weights  of  ( -f  , -^  )  for  survivability  and  firepower, 

respectively,  assigns  the  same  value  to  survivability  of 
applique  armor  and  firepower  of  a  105mm  main  gun.  We 
illustrate  this  concept  by  a  simple  example. 

We  assume  the  decision  maker  has  a  tank  with 
attributes  of  applique  armor  and  a  105mm  main  gun,  and  could 
improve  attributes.  His  or  her  priority  for  doing  so  would  be 
equal  to  the  weights  of  the  attributes.  That  is,  the  decision 
maker  would  have  preference  for  improving  firepower  of  1.25 
times  his  or  her  preference  for  improving  survivability.  This 
illustrates  a  serious  flaw  with  AHP. 

The  weights  of  attributes  cannot  be  independent  of 
alternatives.  In  our  case,  would  the  decision  maker  have  the 
same  priorities  given  a  tank  with  rolled  homogeneous  armor  and 
a  130mm  main  gun?  The  decision  maker  possessing  a  tank  with 
the  lowest  survivability  attribute  (rolled  homogeneous  armor) 
and  the  highest  firepower  attribute  (130mm  main  gun)  should  be 
more  concerned  about  improving  survivability. 

In  this  modification,  the  survivability  rating 
could  be  tripled  by  replacing  the  rolled  homogeneous  armor 
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with  reactive  armor.  This  will  contribute  of  the 

survivability  improvement  to  the  tank's  relative  value. 
Although  the  firepower  attribute  will  contribute  of  its 

improvement  to  the  relative  value  of  the  tank,  the  firepower 
rating  is  already  the  highest  obtainable.  It  would  almost 
certainly  be  nearly  impossible  to  triple  the  firepower  rating. 
Therefore,  improving  the  survivability  attribute  would 
contribute  a  greater  increase  to  the  relative  value  of  the 
tank  than  increasing  firepower.  Despite  the  lower  weight  for 
the  survivability  attribute,  a  decision  maker  will  elect  to 
improve  survivability  over  firepower  if  he  or  she  possessed  a 
tank  with  a  high  firepower  rating  and  low  survivability 
rating. 

With  this  modification,  weights  are  not  independent 
of  alternatives.  Unfortunately,  AHP  cannot  make  this  claim. 
We  can  now  consider  our  units  of  measurement  to  be  in  terms  of 
applique  armor  and  a  105mm  main  gun.  We  will  give  this  "unit" 
a  label  of  SAPF105 . 

b.  First  Approach 

First,  of  the  original  alternatives  considered,  the 
lowest,  or  least  desirable  alternative  is  given  a  rating  of 
one  in  one  of  the  attributes  considered.  That  alternative 
becomes  the  standard  to  determine  attribute  values  for  all 
other  attributes,  and  we  call  that  the  "standard  attribute." 


Attribute  weights  are  normalized  in  a  way  that  this 
modification  does  not  change  (that  is,  the  sum  of  the  weights 
of  the  attributes  will  still  be  equal  to  one)  .  In  other 
attributes,  ratings  could  be  much  lower  than  one.  These 
ratings,  or  scores,  will  depend  on  the  "standard  attribute" 
and  will  be  scaled  accordingly.  An  additional  alternative 
could  be  even  less  preferred  in  an  attribute  than  the 
originally  least  preferred  alternative,  so  it  is  possible  to 
have  a  value  of  less  than  one  in  the  attribute  containing  the 
"standard  attribute"  once  additional  alternatives  are 
considered. 

c .  Second  Approach 

The  second  approach  rates  the  highest,  or  most 
preferable  alternative  in  one  of  the  attributes  as  one.  Once 
additional  attributes  are  considered,  relative  values  of 
higher  than  one  are  possible  in  the  "standard  attribute." 
This  could  occur  if  the  new  alternative  had  a  more  preferable 
rating  in  the  "standard  attribute"  than  the  original 
formulation's  most  preferable  alternative  in  the  "standard 
attribute . " 

2 .  Tank  Problem  Solved  by  First  Approach 

This  section  solves  the  tank  problem  introduced 
earlier  to  demonstrate  rank  reversal  in  a  way  that  avoids  rank 
reversals . 
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a.  Original  Formulation 


The  original  two  tanks  are  evaluated  in  Figure  5 
using  the  approach  that  gives  the  tank  with  the  least 
desirable  characteristic  in  one  attribute  a  score  of  one.  In 
this  case,  survivability  of  Tank  1  is  the  "standard  attribute" 
from  which  other  values  are  derived.  The  decision  maker  has 
determined  equivalency  between  survivability  characteristic  of 
applique  armor  and  firepower  of  a  105mm  main  gun.  The  result 
is  a  different  relative  value  for  each  tank,  but  still  Tank  l 
>  Tank  2 . 


TANK 

ATTRIBUTES 

RELATIVE 

VALUE 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

1.0 

1.5 

1.2778 

TANK  2 

1.5 

1.0 

1.2222 

Figure  5.  First  approach  with  original  formulation. 


The  relative  value  scores  are  not  meaningless.  The 
relative  value  of  a  tank  is  in  terms  of  survivability  of  Tank 
1  and  firepower  of  Tank  2,  because  Tank  2  happens  to  have  a 
firepower  rating  of  1.0.  Its  firepower  attribute,  a  105mm 
main  gun  is  equivalent  to  the  "standard  attribute."  A 
theoretical  "worst"  tank  with  Tank  l's  survivability  (applique 
armor)  and  Tank  2's  firepower  (105mm  gun)  would  receive  a 
relative  value  of  1.0.  Tank  l's  relative  value  of  1.28  means 
it  is  1.28  times  more  desirable  than  this  theoretical  tank. 
Recall  that  we  developed  the  label  SAPF105  for  the  "units"  of 
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measurement  of  tanks  in  terms  of  survivability  attribute 
applique  armor  and  firepower  attribute  of  105mm  main  gun. 
Tank  1  is  "worth"  1.28  SAPF105s,  and  Tank  2  is  "worth"  1.22 
SAPF105S. 

b.  Additional  Alternative  Considered 

The  original  two  tanks,  and  the  third  tank,  are 
evaluated  in  Figure  6.  Again,  the  least  desirable 
characteristic  in  one  attribute  (survivability)  receives  a 
score  of  one.  Once  an  additional  alternative  is  considered, 
it  is  possible  for  an  alternative  to  receive  a  relative  value 
of  less  than  one  in  survivability.  This  could  happen  if  the 
new  alternative  had  attributes  less  desirable  than  the 
original  problem's  least  preferable  alternative  in 
survivability  (Tank  1)  .  The  result  of  considering  the 
additional  alternative  is  the  same  relative  value  scores  for 
the  first  two  tanks. 


TANK 

ATTRIBUTES 

RELATIVE 

VALUE 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

1.0 

1.5 

1.2778 

TANK  2 

1.5 

1.0 

1.2222 

|  TANK  3 

0.5 

2.1 

1.3889 

Figure  6.  First  approach  with  third  tank. 


Tank  3  has  the  highest  relative  value.  Therefore, 


the  decision  maker's  preference  would  be  Tank  3  >  Tank  1  > 
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Tank  2.  There  was  no  rank  reversal  between  the  first  two 


tanks . 

Unlike  AHP,  this  modification 

(i)  did  not  have  rank  reversal, 

(ii)  maintained  a  ratio  scale  of  relative  value. 

c.  Duplication  of  Alternative 

A  fourth  tank  is  now  added.  Just  as  in  the 


previous  example,  it  has  the  same  characteristics  as  Tank  2. 
These  four  tanks  are  evaluated  in  Figure  7. 


TANK 

ATTRIBUTES 

RELATIVE  I 

VALUE  S 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

1.0 

1.5 

1.2778 

TANK  2 

1.5 

1.0 

1.2222 

TANK  3 

0.5 

2.11 

1.3889 

TANK  4 

1.5 

1.0 

1.2222 

Figure  7 .  First  approach  with  fourth 

The  first  three  tanks  receive  the  same  relative 


value  scores  as  they  did  when  considered  previously.  The 
preferences  for  these  r"iur  tanks  are  Tank  3  >  Tank  l  >  Tank  2 
=  Tank  4 .  There  is  no  rank  reversal  among  the  original 
alternatives . 


Adding  the  fourth  tank 

(i)  did  not  cause  rank  reversal, 

(ii)  maintained  a  ratio  scale  of  relative  value. 
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d.  A nalysia 

If  we  let  R  aT  be  the  pairwise  comparison  matrix 

obtained  by  pairwise  comparisons  of  the  relative  values  of  the 
two  tanks,  we  obtain  the  following  matrix: 

Tl  T2 
1  1.05 

0.96  1 

If  we  let  R  \T  be  the  pairwise  comparison  matrix  obtained  by 

the  pairwise  comparisons  of  the  relative  values  of  the  four 
tanks,  we  obtain  the  following  matrix: 


R 


2T 


Tl 

T2 


Tl 

Tl 

1 

T2 

1.05 

T3 

0.92 

T4 

1.05 

T2 

0.96 

1 

0.88 

1 

T3 

1.09 

1 . 14 

1 

1 . 14 

T4 

0.96 

1 

0.88 

1 

The  upper  left  2x2  entries  of  the  matrix  are  identical  to  the 
matrix  R  '2T.  Therefore,  the  ratio  scale  was  maintained.  Rank 

reversal  will  not  occur  because  the  relative  value  of 
alternatives  will  not  change. 

3 .  Tank  Problem  Solved  by  Second  Approach 

This  section  solves  the  tank  problem  using  the  second 
approach  described  earlier.  The  highest  or  most  preferred 
alternative  in  one  attribute  receives  a  score  of  one.  In  this 
case,  we  select  Tank  2's  survivability  attribute  as  the 
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"standard  attribute"  by  which  other  alternatives  within  that 
attribute  will  be  measured.  Let  us  again  assume  that  the 
decision  maker,  in  order  to  "link"  the  two  attributes,  has 
determined  indifference  between  Tank  l's  survivability 
(applique  armor)  and  the  firepower  of  Tank  2  (105mm  main  gun)  . 
Since  this  is  the  same  equivalency  established  in  the  first 
method,  we  should  expect  to  maintain  the  same  ratio  scale  for 
this  method,  if  it  is  to  have  merit, 
a.  Original  Formulation 

The  original  two  tanks  are  evaluated  in  Figure  8 
using  the  second  approach.  The  result  is  a  different  relative 
value  score  for  each  tank,  but  still  Tank  1  >  Tank  2. 

Essentially,  what  we  have  done  is  change  "units."  Now  tanks 
are  evaluated  on  the  basis  of  Tank  2's  survivability  (reactive 
armor)  and  Tank  l's  firepower  (120mm  main  gun)  .  We  will  label 
these  "units"  SRAF120 . 


TANK 

ATTRIBUTES 

RELATIVE 

VALUE 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

0.67 

1.0 

0.8519 

TANK  2 

0 . 67 

0.8148 

The  relative  values  obtained  by  this  method  have  meaning.  A 
theoretical  "best"  tank  with  the  highest  possible  rating  in 
each  attribute  (this  would  be  a  tank  with  reactive  armor  and 
a  120mm  gun)  would  receive  a  score  of  one.  Tank  l's  relative 
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value  of  0.85  means  it  is  0.85  times  as  desirable  as  this 
theoretical  tank.  From  units  derived  above,  Tank  1  is  worth 
0.85  SRAF120S. 

b.  Additional  Alternative  Considered 

The  original  two  tanks,  and  the  third  tank,  are 
evaluated  in  Figure  9.  Again,  the  most  desirable 
characteristic  in  one  attribute  (survivability)  receives  a 
score  of  one.  It  is  now  possible  for  the  new  alternative 
(Tank  3)  to  receive  a  relative  value  score  of  greater  than  one 
in  survivability.  A  rating  of  one  is  based  on  survivability 
of  reactive  armor.  The  result  of  considering  the  additional 
alternative  is  the  same  relative  value  scores  for  the  first 
two  tanks. 


TANK 

ATTRIBUTES 

RELATIVE 

VALUE 

SURVIVABILITY 

(4/9) 

FIREPOWER 

(5/9) 

TANK  1 

0.67 

1.0 

0.8518 

TANK  2 

1.0 

0.67 

0.8148 

TANK  3 

0.33 

1.4 

0.9259 

Figure  9 . Second  approach  with  third  tank. 

This  method  also  avoided  the  rank  reversal  encountered  in  the 
original  AHP  formulation  of  this  problem. 


This  modification 

(i)  avoided  the  rank  reversal  of  AHP, 

(ii)  maintained  a  ratio  scale  of  relative  value. 
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c.  Duplication  of  Alternative 

A  fourth  tank,  with  the  same  characteristics  as 
Tank  2  is  added.  These  four  tanks  are  evaluated  in  Figure  10. 


Figure  10. Second  approach  with  fourth  tank. 

Unlike  the  example  of  using  AHP  with  an  additional  alternative 
added  having  the  same  characteristics  as  an  original 
alternative,  this  method  avoids  rank  reversal.  The  reason  for 
this  is  the  relative  value  of  an  alternative  will  not  change 
unless  the  decision  maker  makes  a  conscious  decision  to  change 
weights  of  alternatives  or  the  rating  of  an  alternative 
because  of  reassessment  or  new  information. 

Adding  the  fourth  tank 
(i)  did  not  cause  rank  reversal, 

(ii)  maintained  a  ratio  scale  of  relative  value. 

d.  Analysis 

If  we  let  R~2t  be  t^e  pairwise  comparison  matrix 

obtained  by  pairwise  comparisons  of  the  relative  values  of  the 
two  tanks  in  the  original  problem,  we  obtain  the  following 
matrix: 
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Tl  T2 
Tl  1  1.05 

R  = 

2T  T2  0.96  1 

If  we  let  H"4r  be  the  pairwise  comparison  matrix  obtained  by 

pairwise  comparisons  of  the  relative  values  of  the  four  tanks 
using  the  second  approach,  we  obtain  the  following  matrix: 


Tl 

T2 

T3 

T4 

Tl 

1 

1.05 

0.92 

1.05 

T2 

0.96 

1 

0 . 88 

1 

T3 

1.09 

1.14 

1 

1 . 14 

T4 

0.96 

1 

0.88 

1 

The  upper  left  2x2  matrix  is  identical  to  the  2x2  matrix 
obtained  in  the  original  formulation  from  R~2r-  This  shows 

that  this  approach  maintains  a  ratio  scale  when  additional 
alternatives  are  considered. 

4.  Conclusion 

Both  approaches  maintain  a  ratio  acala  of  alternatives 
and  avoid  rank  reversals.  Because  R  \T-R~€T,  either  approach 

solves  the  rank  reversal  problems  identified  previously  when 
using  AHP.  Whether  the  decision  maker  decides  to  rank  the 
highest  alternatives  in  one  attribute  as  one,  or  the  lowest  as 
one,  he  or  she  will  obtain  identical  answers.  That  is,  the 
hierarchy  of  preferences  will  be  identical  and  the  ratios  of 
comparisons  of  alternate's  will  be  identical. 
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V.  CONCLUSION 


There  are  at  least  three  modifications  which  should  be 
made  to  AHP  before  it  is  used  in  a  decision  making  problem. 

A.  PAIRWISE  COMPARISON  SCALING 

The  nine  point  scale  used  in  AHP  is  overly  restrictive. 
There  are  at  least  two  ways  to  modify  scaling  procedures  that 
eliminate  a  significant  potential  for  inconsistency. 

1.  Decision  Maker  Defined  Scale 

Instead  of  using  the  value  of  nine  to  signify  extreme 
importance  of  one  attribute  over  another  (or  one  element  of  an 
alternative  over  another) ,  the  decision  maker  should  define 
the  nine  point  scale  based  on  the  characteristics  of  the 
problem.  We  suggest  defining  the  most  preferred  alternative 
as  nine,  and  the  least  preferred  alternative  as  one.  All 
intermediate  alternatives  must  fall  within  the  scale. 
Similarly,  when  making  pairwise  comparisons  of  alternatives  by 
attribute,  the  extreme  alternatives  should  be  used  to  define 
the  scale. 

2.  Open  Ended  Scale 

The  decision  maker  could  alternatively  use  an  open 
ended  scale  for  pairwise  comparisons.  If  the  decision  maker 
feels  there  are  several  orders  of  magnitude  of  difference 
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between  alternatives,  he  or  she  could  use  a  multiplicative 
open  ended  scale  to  determine  preferences. 

B .  WEIGHT  PITTING 

There  appears  to  be  no  advantage  to  using  AHP's  eigenvalue 
method  of  weight  fitting,  other  than  its  ease  of  evaluation  by 
the  AHP  consistency  index.  But  since  the  consistency  index 
standard  of  0.9  is  itself  arbitrary,  evaluating  one's  decision 
making  by  the  consistency  index  is,  in  reality,  not  much  of  an 
advantage.  The  least  square  error  (LSE)  method  of  weight 
fitting,  not  surprisingly,  produces  a  smaller  error.  Of 
course,  LSE  was  used  to  determine  the  error  of  both  methods. 

C.  RANK  REVERSAL 

We  have  demonstrated  two  ways  to  avoid  rank  reversals. 
Both  methods  depend  on  the  decision  maker  to  establish  and 
maintain  a  system  of  "units"  in  which  relative  values  of 
alternatives  will  be  measured. 

1.  First  Method 

The  decision  maker  selects  the  least  preferred 
alternative  in  one  of  the  attributes  considered.  This  least 
preferred  alternative,  which  we  call  the  "standard  attribute" 
is  assigned  a  value  of  one.  All  other  alternatives  within 
that  attribute  are  assigned  values  based  on  each  alternative's 
comparison  to  the  least  preferred  alternative.  The  decision 
maker  must  also  relate  the  value  of  the  "standard  attribute" 
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to  other  attributes.  This  is  an  essential  step  to  maintaining 
a  ratio  scale.  The  decision  maker  will  decide  a  ratio  between 
the  "standard  attribute"  and  an  element  within  each  of  the 
other  attributes.  No  matter  how  the  problem  is  changed  from 
this  point,  that  ratio  must  remain  the  same.  The  alternatives 
must  not  be  renormalized  with  the  addition  or  deletion  of 
alternatives . 

2 .  Second  Method 

The  decision  maker  selects  the  most  preferred 
alternative  in  one  of  the  attributes  considered.  This  most 
preferred  alternative,  which  is  the  "standard  attribute"  is 
assigned  a  value  of  one.  All  other  alternatives  within  that 
attribute  are  assigned  values  based  on  each  alternative's 
comparison  to  the  most  preferred  alternative.  The  decision 
maker  conducts  identical  steps  as  performed  in  the  first 
method  to  relate  values  to  other  attributes. 

3.  Results 

Both  methods  produced  identical  pairwise  comparison 
matrices  at  every  step  of  the  tank  example  problem.  Neither 
method  will  generate  rank  reversals  when  the  decision  maker  is 
consistent.  We  demonstrated  use  of  invented  units  as  labels 
for  the  magnitude  of  the  relative  value  in  each  example  of  the 
use  of  each  method.  Since  the  notional  units  were 
consistently  used  to  measure  the  relative  value  of 
alternatives  as  additional  alternatives  were  considered,  the 
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relative  value  of  alternatives  never  changed.  Both  methods 
avoid  rank  reversals  and  maintain  a  ratio  scale. 
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APPENDIX  A 


AHP  AND  CHOOSING  A  HOUSE 

This  example  is  taken  from  T.  L.  Saaty,  "How  to  make  a  decision: 
The  Analytic  Hierarchy  Process* ,  European  Journal  of  Operations  Re¬ 
search,  Vol .  48,  pages  9  -  26,  1990.  The  purpose  is  to  compare  three 
houses.  A,  B,  and  C,  using  eight  attributes  to  find  the  most  pre¬ 
ferred  house. 


The  following  matrix  shows  a  pairwise  comparison  of  all  eight  (8) 
attributes,  together  with  the  vector  of  weights  produced  by  AHP 
that  add  to  1.0. 


2 


3 


8 


PRIORITY 


VECTOR 
.173 

.054 

.188 

.018 

.031 

.036 

.167 

.333 

A  Matrix  and  Priority  Vector  for  House  Ranking  Problem 

The  following  eight  (8)  matrices  show  the  results  of  pairwise  com¬ 
parisons  of  the  three  houses  on  each  attribute. 


1 


SIZE 

A 

B 

C 

PRIORITY  VECTOR 

A 

1 

6 

8 

.754 

B 

1/6 

1 

4 

.181 

C 

1/8 

1/4 

1 

.065 

EV=3  . 

136 

YARD 

SPACE 

A 

B 

C 

PRIORITY  VECTOR 

A 

1 

5 

4 

.674 

B 

1/5 

1 

1/3 

.101 

C 

1/4 

3 

1 

.226 

EV=3  . 

068 
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TRANSPORTATION 


ABC 
A  1  7  1/5 

B  1/7  1  1/8 

C  5  8  1 

EV=3 .247 

MODERN  FACILITIES 
ABC 
A  1  8  6 

B  1/8  1  1/5 

C  1/65  1 

EV=3 .197 
NEIGHBORHOOD 

ABC 
A  1  8  6 

B  1/8  1  1/4 

C  1/64  1 

EV=3 .136 

GENERAL  CONDITION 
ABC 
A  1  1/2  1/2 

B  2  1  1 

C  2  1  1 

EV=3 .000 
AGE  OF  HOUSE 

ABC 
A  1  1  1 

Bill 
C  1  1  1 

EV=3 .000 
FINANCING 

ABC 
A  1  1/7  1/5 

B  7  1  3 

C  5  1/31 

EV=3 .065 


PRIORITY  VECTOR 
.233 
.054 
.713 


PRIORITY  VECTOR 
.747 
.060 
.193 


PRIORITY  VECTOR 
.745 
.065 
.181 


PRIORITY  VECTOR 
.200 
.400 
.400 


PRIORITY  VECTOR 
.333 
.333 
.333 


PRIORITY  VECTOR 
.072 
.650 
.278 
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LOCAL  AND  GLOBAL  PRIORITIES 


12345678 
(  .173)  ( .054)  ( .188)  (  .018)  ( .031)  ( .036)  ( .167)  (  .333) 


A 

.754 

.233 

.754 

.333 

B 

.181 

.055 

.065 

.333 

C 

.065 

.713 

.181 

.333 

.674 

.747 

.200 

.072 

.396 

.101 

.060 

.400 

.650 

.341 

.226 

.193 

.400 

.278 

.263 

From  these  AHP  shows  that  House  A  is  preferred  to  B  is  preferred 
to  C. 
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APPENDIX  B 


What  follows  is  a  demonstration  of  how  to  determine  the 
least  squares  approximation  of  determining  weights  of 
attributes.  This  demonstration  was  performed  on  a  3x3  matrix, 
but  the  reader  should  be  able  to  recognize  the  pattern  that 
develops . 

Recall  that  if  the  pairwise  comparisons  are  perfectly 
consistent,  the  following  relationship  holds. 

<*>j  .... 

rii  = — -  ,  rii  given  m  pairwise  comparisons . 

J  <0  j  J 

The  error  for  the  comparisons 

e(u)  =  £  £  (ui-G>,r.M) 2 . 

i* 1  j* l 

By  definition,  main  diagonal  entries  must  be  one,  or 
rii  =  l  Vi.  Expanding  the  error  term  for  a  3x3  matrix 

e  =  (<o1-w2r12)2+ (w1-a)3r13)2+ (w2-w1r21)2  + 

(w2  -w3r23) 2  +  (q3  —  G)3r33 ) 2  +  (<*>3  ~<*>2r32) 2  . 

The  reason  the  above  expression  contains  only  six  terms  is 
because  the  entries  for  i=j  are  all  zero. 

The  row  entries  of  the  solution  matrix  are  the  respective 
partial  derivatives  of  the  error  with  respect  to  ,  where  i 

designates  the  row  number,  and  setting  the  equation  equal  to 
zero.  For  the  first  row, 
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=2  (g>!-< o2r12)  ♦ 2  (u3-u3r13)  -2 r21  (w2-o>1r21)  -2r31  (w3-o>1r31) 

2w,  =  -o3r212  -o3r312  +o>2  (r12-r21)  +  u3  (r31  -r13) 

3«1=  “x-Wi^2i2-«i^3i2+w2(  ^2-^21)  +“3(r3i--ri3)  • 

The  last  step,  adding  u1  to  each  side,  was  done  in  light  of 

the  solution  we  are  seeking,  Eu  =  nu,  where  E  is  the  matrix  of 
errors,  u  is  the  vector  of  weights  and  n  is  the  dimension  of 
this  vector.  Similarly,  the  partial  derivatives  with  respect 
to  g>2  and  0 >3  can  be  determined,  which  lead  to  the  following 

matrix  representing  the  solution  form  of  matrix  E. 


(l-r212-r312) 

^12+^2l) 

^i3  +  -r3i) 

(ri2  +  r21) 

(l-r122-r322) 

^23  +r32) 

<ri3  +  ^31> 

^23+-r32) 

^  3-  "  ri32  ”  r23 

To  determine  the  error  when  using  the  dimension  method,  we 
solve  the  equation 

Xu  =  nu  . 

Recall  that  in  the  case  of  inconsistency,  there  is  no  exact 
solution  to  this  equation.  There  are  many  ways  to  approximate 
the  vector  «.  The  procedure  for  deriving  the  dimension 
method's  approximation,  obtained  by  multiplying  both  sides  by 
an  identity  matrix  of  size  n 
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Xu  =  nlu 


(nX-R)  «  =  0  . 


Expanding, 


This  is  equivalent  to: 

i~2  ?ijUj 

j*  i 

nu2-Er2u-  =0. 

j-i 

This  leads  to  a  final  expression  for  the  error  matrix  for  the 
dimension  method  of  estimating  weights  shown  below 
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APPENDIX  C 


SIZEM  COMPARE  WSSIZE 
METHOD  ERROR  WEIGHTS 

SAATY  :  17.820  .754  .181  .065 
DIMENSION  :  12.314  .727  .202  .071 
LEAST  SQRS:  7.477  .780  .136  .084 

TRANSM  COMPARE  WSTRANS 
METHOD  ERROR  WEIGHTS 

SAATY  :  35.902  .233  .055  .712 
DIMENSION  :  33.357  .199  .060  .740 
LEAST  SQRS:  29.833  .193  .064  .743 

NEIGHM  COMPARE  WSNEIGH 
METHOD  ERROR  WEIGHTS 

SAATY  :  17.820  .754  .065  .181 
DIMENSION  :  12.314  .727  .071  .202 
LEAST  SQRS:  7.477  .780  .084  .136 

AGEM  COMPARE  WSAGE 
METHOD  ERROR  WEIGHTS 

SAATY  :  .000  .333  .333  .333 
DIMENSION  :  .000  .333  .333  .333 
LEAST  SQRS  :  .000  .333  .333  .333 

YARDM  COMPARE  WSYARD 
METHOD  ERROR  WEIGHTS 

SAATY  :  4.439  .674  .101  .226 
DIMENSION  :  3.627  .655  .105  .240 
LEAST  SQRS:  3.026  .697  .118  .185 
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MODFACM  COMPARE  WSMODFAC 
METHOD  ERROR  WEIGHTS 
SAATY  :  27.540  .747  .060  .193 
DIMENSION  :  17.584  .707  .067  .226 
LEAST  SQRS:  14.946  .784  .078  .138 

GENCONM  COMPARE  WSGENCON 
METHOD  ERROR  WEIGHTS 
SAATY  :  .000  .200  .400  .400 
DIMENSION  :  .000  .200  .400  .400 
LEAST  SQRS  :  .000  .200  .400  .400 
FINANCEM  COMPARE  WSFINANCE 
METHOD  ERROR  WEIGHTS 
SAATY  :  5.860  .072  .650  .278 
DIMENSION  :  8.334  .068  .654  .278 


LEAST  SQRS  :  5  . 

438  .087 

.670  ., 

243 

HOUS EATTM 

1.000  5.000 

3.000 

7.000 

6.000 

6.000 

0.333  0.250 

0.200  1.000 

0.333 

5.000 

3.000 

3.000 

0.200  0.143 

0.333  3.000 

1.000 

6.000 

3.000 

4.000 

6.000  0.200 

0.143  0.200 

0.167 

1.000 

0.333 

0.250 

0.143  0.125 

0.167  0.333 

0.333 

3.000 

1.000 

0.500 

0.200  0.167 

0.167  0.333 

0.250 

4.000 

2.000 

1.000 

0.200  0.167 

3.000  5.000 

0.167 

7.000 

5.000 

5.000 

1.000  0.500 

4.000  7.000 

5.000 

8.000 

6.000 

6.000 

2.000  1.000 

HOUSEATTM  COMPARE  WSHOUSEATT 

METHOD  ERROR 

WEIGHTS 

SAATY: 259 

.173  .054  .188 

.018 

.031 

.036  .167  .333 

DIMENSION: 858  ■ 

-.031  .081  .258 

.022 

.043 

.052  .170  .404 

LSE : 237 

.279  .447  .098 

.027 

.041 

.041  .072  .395 

65 


LIST  OF  REFERENCES 


1.  Saaty,  Thomas  L.  ,  How  to  make  a  decision:  The  Analytic 

Hierarchy  Process,  European  Journal  of  Operational 
Research,  Vol .  48,  (1990)  9-26. 

2.  Schoner,  Bertram  and  Wedley,  William  C.,  Ambiguous 

Criteria  Weights  in  AHP:  Consequences  and  Solutions, 

Decision  Sciences,  Vol.  20,  (1989)  462-475. 

3.  Howard,  Ronald  A.,  Heathens,  Heretics  and  Cults:  The 

Religious  Spectrum  of  Decision  Aiding,  Interfaces,  Vol. 
22,  No.  6,  (1992)  15-27. 

4.  Holder,  R.  D.,  Some  Comments  on  the  Analytic  Hierarchy 

Process,  Journal  of  Operational  Research  Society,  Vol.  41, 
No.  11,  (1990)  1073-1076. 

5.  Roper-Lowe,  G.  C.  and  Sharp,  J.  A.,  The  Analytic  Hierarchy 

Process  and  its  Application  to  Information  Technology 
Decision,  Journal  of  Operational  Research  Society,  Vol. 
41,  (1990)  49-59. 

6.  Belton,  V.  and  Gear,  T. ,  On  a  shortcoming  of  Saaty’ s 

Method  of  Analytic  Hierarchies,  Omega,  Vol.  11,  (1983) 

228-230. 

7.  Hihn,  Jairus  M.  ,  and  Johnson,  Charles  R.,  Evaluation 
Techniques  for  Paired  Ratio-Comparison  Matrices  in  a 
Hierarchical  Decision  Model,  Measurement  in  Economics, 
Physica- Verlag  Heidelberg,  (1988)  269-288. 

8.  Saaty,  Thomas  L.  and  Vargas,  L.  G.,  Experiments  on  Rank 

Preservation  and  Reversal  in  Relative  Measurement, 
Mathematical  Computational  Modeling,  Vol.  17,  (1993)  13- 

18. 


66 


INITIAL  DISTRIBUTION  LIST 

No.  Copies 


1.  Defense  Technical  Information  Center  2 

Cameron  Station 

Alexandria  VA  22304-6145 

2.  Library,  Code  052  2 

Naval  Postgraduate  School 

Monterey  CA  93943-5002 

3.  Professor  Kneale  T.  Marshall,  Code  OR/Mt  5 

Department  of  Operations  Research 

Naval  Postgraduate  School 
Monterey,  CA  93943-5000 

4.  Professor  William  G.  Kemple,  Code  OR/Ke  2 

Department  of  Operations  Research 

Naval  Postgraduate  School 
Monterey,  CA  93943-5000 

5.  Professor  David  R.  Whipple,  Jr.,  Code  AS/Wp  2 

Department  of  Administrative  Sciences 

Naval  Postgraduate  School 
Monterey,  CA  93943-5000 

6.  Professor  Thomas  C.  Bruneau,  Code  NS/Bn  2 

Department  of  National  Security  Affairs 

Naval  Postgraduate  School 
Monterey,  CA  93943-5100 

7.  Professor  Paul  H.  Moose,  Code  CC  2 


Department  of  Command,  Control  and  Communication 
Naval  Postgraduate  School 
Monterey,  CA  93943-5000 

8.  Dr.  Michael  R.  Anderson  2 

Operations  Research  Analyst 

Combined  Arms  Analysis  Directorate,  TRAC-OAC 
Fort  Leavenworth,  KS  66027 

9.  CPT  William  H.  McQuail  2 

6432  Woodville  Dr. 

Falls  Church,  VA  22044-1429 


67 


